The modern operating system (OS) provides a graphical user interface (GUI) for interacting with the user interface (UI) elements present therein. The GUI makes use of a spatial selection paradigm in which navigating the GUI involves mapping movements that are detected using some input device (e.g., a mouse) to the virtual interface of the GUI in order to interact with the user interface (UI) elements of the GUI. As computers and input devices have evolved from keyboards and mice to modern day multi-touch trackpads and multi-touch touchscreen surfaces, the GUI has been adapted to maintain compatibility with legacy input devices (i.e., keyboard and mouse) while also incorporating benefits that are provided by the newer input devices. In so doing however, the OS and/or GUI has mutated in a way such that no single input device or input paradigm is singularly suited to fully interact with the OS and/or GUI because each input device has its own set of advantages and disadvantages.
The keyboard is a discrete input device wherein each key of the keyboard can be used to provide a discrete input. The keyboard is efficient for use in performing text entry, but inefficient for use in navigating the spatial selection paradigm of the GUI. While keyboard “shortcuts” which utilize “flag keys (e.g., option, control, command, alt, etc.) provide access to certain elements of a GUI, such keyboard shortcuts are often difficult to remember, inconsistent across different applications, and to invoke a keyboard shortcut, the user must often perform awkward hand movements and finger extensions in order to reach flag keys that are used in combination with other keys. As a result, the keyboard remains the preferred input device when performing operations that require discrete input (e.g., text entry), but is deficient when navigating and utilizing other aspects of the GUI.
In direct contrast to the keyboard, pointer based input devices, such as a mouse or trackpad, are useful for navigating a GUI and selecting and invoking user interface (UI) elements therein, but are inefficient for use in performing text entry. Consequently, to operate a modern day computer system or device, many users rely on the functionality of a keyboard used in conjunction with a pointer based input device.
Touchscreen based input devices have attempted to bridge the gap between discrete input devices and spatially dependent input devices. Specifically, touchscreen based input devices allow users to directly touch and thereby manipulate UI elements that are perceived on the device, which provides much of the traditional pointer based functionality. To then function as a keyboard, devices that utilize touchscreen based interaction provide virtual keyboards that are displayed on-screen in conjunction with various touch based gestures that assume the role of text entry. However, for intensive text composition, coding, game playing, or other common computer activities where discrete and/or precise input is required, a secondary input device such as the keyboard is often used to overcome the deficiencies and lack of tactical feedback of the touchscreen based input device. Furthermore, precise manipulation is challenging on touch based devices, because the user's fingers necessarily obstruct the exact area where the manipulation is desired. Also, there are ergonomic limitations to prolonged usage when either the arm is held up to the screen, or the neck is strained by looking down at the screen.
Accordingly, there is a need to better adapt existing input functionality of one or more of these input devices to bridge the divide in navigating a GUI and performing text entry and other input that requires precision or discrete interaction. Stated differently, there is a need to integrate functionality from pointer based UIs, direct touch UIs, and discrete input UIs to update the capabilities of the GUI and modern day OS software to better adapt to various input device hardware advancements.